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DETAILED ACTION 



Response to Amendment 



1 . The declaration filed on April 09, 2009 under 37 CFR 1.131 is sufficient to 
overcome the Lou et al. reference. Upon closer inspection of the Lou et al. reference, it 
is noted that it was publicly presented on August 14, 2002, a day after the application 
papers were prepared and the same day of its mailing to the British Patent Office. 
However, upon further consideration, a new grounds of rejection is made in view of 
newly found prior art references. 

2. This Office Action is responsive to Applicant's remarks received on April 09, 
2009. Claims 1-15 remain pending. 



Claim Objections 



3. The following is a quotation of 37 CFR 1 .75(a): 

The specification must conclude with a claim particularly pointing out and distinctly claiming the subject 
matter which the applicant regards as his invention or discovery. 

4. Claims 1 and 1 5 are objected to under 37 CFR 1 .75(a), as failing to particularly 
point out and distinctly claim the subject matter which application regards as his 
invention or discovery. 
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Claim 1 lacks antecedent basis for "the image" at line 17. The following will be 
assumed for examination purposes: -- the i mago video images --. The same applies to 
claim 15 at line 6. 

Claim 15 lacks antecedent basis for "the video image" at line 18. The following 
will be assumed for examination purposes: - the video i mag e images "--. 



Claim Rejections - 35 USC § 101 

5. The previous 101 rejections have been withdrawn in light of Applicant's 
amendment. 



Claim Rejections - 35 USC § 103 



6. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

7. Claims 1,2, 9-12, 14 and 15 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over the combination of Owens et al. ("Application of the Self-Organising 
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Map...", IEEE Article 2000, hereinafter Owens 2000) and Owens et al. ("Novelty 
Detection in Video Surveillance...", Springer Article 2002, hereinafter Owens 2002). 

Regarding claim 1, Owens 2000 discloses a method for a video motion anomaly 
detector for processing video images to detect an event of interest ("detecting 
suspicious behavior of pedestrians using a computerised grey-scale video surveillance 
system" at page 1 , section 1 , line 1 ), comprising: 

receiving, by the video motion anomaly detector, a video signal representing the 
video images to be processed ("video sequences were recorded" at page 3, section 4, 
line 4); 

extracting, by the video motion anomaly detector, at least one point feature from 
the video signal ("sequence of centroid points for each tracked object" at page 3, 
section 2, line 2); 

tracking, by the video motion anomaly detector, the position and movement of 
the at least one point feature within the video images to generate a corresponding at 
least one track, each of said at least one track representing a corresponding point 
feature in terms of its position and its velocity within each of the video images ("flow 
vector, f = [x, y, dx. dy]" at page 3, section 4, paragraph 3, line 1 ; "Each individual point 
in the trajectory is translated into a feature vector" at page 5, section 4, last sentence); 

using, by the video motion anomaly detector, an iterative learning process to 
derive a normal pattern of behavior for each track for each position within the video 
images in terms of observed incidences of point feature velocity at said each position 
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("training data consisted of 206 normal trajectories" at page 6, section 6, paragraph 2, 
line 5); 

comparing present behavior of a track at a certain position within the image to 
the respective derived normal pattern of behavior at the certain position in terms of 
observed point feature velocity at the certain position ("test set included 23 unusual 
trajectories and 16 normal trajectories" at page 6, section 6, paragraph 2, line 6; 
"complete trajectory was classified as unusual if it contained two or more points 
classified as unusual" at page 6, section 6, paragraph 4; "novel points are instances in 
which a trajectory segment was either in an unfamiliar orientation, or of an unfamiliar 
magnitude (i.e. unusual speed)" at page 6, section 6, paragraph 8, line 9); and 

in response to the present behavior falling outside the normal pattern of behavior 
in terms of observed point feature velocity at the certain position, the video motion 
anomaly detector generates a signal (as seen in figure 3a, points that are unusual are 
shown in black, and the trajectory is labeled as unusual). 

Owens 2000 does not explicitly disclose that the signal is an alarm signal. 

Owens 2002 teaches a method for a video motion anomaly detector for 
processing video images to detect an event of interest (see title and figure 3), 
comprising: 

in response to the present behavior falling outside the normal pattern of behavior, 
the video motion anomaly detector generates an alarm signal ("In abnormal sequences 
the novelty accumulator nodes are allowed to increase their activity, generating an 
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alarm state" at abstract, last sentence; "alarm reporting as the event is developing" at 
page 1254, section 4, second paragraph, line 5). 

It would have obvious at the time the invention was made to one of ordinary skill 
in the art to utilize the alarm state signal of Owens 2002 to signify the unusual behavior 
of Owens 2000 such that crime prevention may occur by alerting a surveillance operator 
to the existence of possible suspicious behavior (see Owens 2002 at page 1254, 
second to last paragraph). 

Regarding claim 2, the Owens 2000 and Owens 2002 combination discloses a 
method wherein the alarm signal causes at least one of the following effects: 

draw the attention of the operator ("drawing the attention of a human operator to 
such events" Owens 2000 at page 1 , section 1 , line 6); 

place an index mark at the appropriate place in recorded video data; and 

trigger selective recording of video data. 

Regarding claim 9, the Owens 2000 and Owens 2002 combination discloses a 
method wherein abnormal tracks are filtered ("objective of the system is to act as a filter, 
detecting behavior that appears out of the ordinary" Owens 2000 at page 1 , section 1 , 
line 4), whereby an active alarm signal is generated in response to an abnormal track 
which resembles a number of other abnormal tracks, in terms of at least one of position, 
velocity and time ("pedestrian moving in between parked vehicles shows high rates of 
velocity change as the path reverses direction" Owens 2000 at page 5, section 4, 
paragraph 3, line 8; a trajectory similar to the one in figure 3 would be indicated as 
unusual as it would resemble figure 3 in terms of position and velocity). 
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Regarding claim 10, the Owens 2000 and Owens 2002 combination discloses a 
method wherein abnormal tracks are filtered, whereby an active alarm signal is 
generated in response only to an abnormal track which has been classified as abnormal 
on a predetermined number of occasions (i.e. once). 

Regarding claim 11, the Owens 2000 and Owens 2002 combination discloses a 
method wherein abnormal tracks are filtered, whereby an active alarm signal is 
generated in response only to a track being classified as abnormal for the first time 
(once the trajectory is indicated as unusual and the operator's attention is drawn to it, 
there is no further need to raise an alarm). 

Regarding claim 12, the Owens 2000 and Owens 2002 combination discloses a 
method wherein abnormal tracks are filtered, whereby an active alarm signal is 
generated only in response to a filtered version of a classification as abnormal rising 
above a predetermined threshold value ("If this distance exceeds a threshold value then 
the feature vector is considered novel, and the trajectory is identified as suspicious" 
Owens 2000 at page 6, section 5, paragraph 7, line 5; "complete trajectory was 
classified as unusual if it contained two or more points classified as unusual" Owens 
2000 at page 6, section 6, paragraph 4). 

Regarding claim 14, the Owens 2000 and Owens 2002 combination does not 
explicitly disclose that subsequent active alarm signals are inhibited if caused by an 
abnormal track within a predetermined distance of another track which has previously 
generated an alarm. 
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However, Owens 2000 suggests that subsequent active alarm signals can be 
inhibited if caused by an abnormal track within a predetermined distance of another 
track which has previously generated an alarm ("One of the three normal trajectories 
classified as unusual, traced a pedestrian moving. ..Upon examination of the training 
set, it was found that a trajectory in this location and direction was only represented 
once in the training set... This implies a requirement to update the neural network online 
with newly-detected normal trajectories when they occur" at page 6, section 6, 
paragraph 6; as such, subsequent unusual trajectories that resemble the initial unusual 
trajectory will not generate an alarm). 

Therefore, it would have been obvious at the time the invention was made to one 
of ordinary skill in the art to utilize the neural network updating as suggested by Owens 
2000 in the system of the Owens 2000 and Owens 2002 combination to overcome the 
problem that "novel but actually acceptable behaviour will be classified as suspicious" 
(Owens 2000 at page 6, section 6, paragraph 6, line 14). 

Regarding claim 15, Owens 2000 discloses an apparatus for processing video 
images to detect an event of interest ("standard PC" at page 7, section 7, paragraph 1, 
last sentence; "detecting suspicious behavior of pedestrians using a computerised grey- 
scale video surveillance system" at page 1, section 1, line 1), comprising: 

a source of video images, which produces a video signal representing the video 
images to be processed ("video sequences were recorded" at page 3, section 4, line 4); 



Application/Control Number: 10/524,554 Page 9 

Art Unit: 2624 

a feature extraction device that receives the video signal, and produces data 
representing at least one point feature detected within the image ("sequence of centroid 
points for each tracked object" at page 3, section 2, line 2); 

a feature tracking device that receives the data representing said at least one 
point feature, and produces data representing a track that is representative of position 
and velocity of each of said at least one point feature within the image ("flow vector, f = 
[x, y, dx. dy]" at page 3, section 4, paragraph 3, line 1; "Each individual point in the 
trajectory is translated into a feature vector" at page 5, section 4, last sentence); 

a learning device that receives the data representing tracks of said at least one 
point feature, and derives a normal range of behavior of each position within the video 
images in terms of observed incidences of point feature velocity at said each position in 
response to operation of a learning process on the data representing the tracks 
("training data consisted of 206 normal trajectories" at page 6, section 6, paragraph 2, 
line 5); 

a classification device that receives both the signal representing the normal 
range of behavior and the data representing the tracks, and is adapted to compare the 
signal and the data for a corresponding position within the video images and to issue a 
normal/abnormal signal in accordance with the outcome of such comparison ("test set 
included 23 unusual trajectories and 16 normal trajectories" at page 6, section 6, 
paragraph 2, line 6; "complete trajectory was classified as unusual if it contained two or 
more points classified as unusual" at page 6, section 6, paragraph 4; "novel points are 
instances in which a trajectory segment was either in an unfamiliar orientation, or of an 
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unfamiliar magnitude (i.e. unusual speed)" at page 6, section 6, paragraph 8, line 9); 
and 

a generation device that receives the normal/abnormal signal and generates at 
least one active signal in response to the normal/abnormal signal indicating abnormal 
behavior of at least one track (as seen in figure 3a, points that are unusual are shown in 
black, and the trajectory is labeled as unusual). 

Owens 2000 does not explicitly disclose that the signal is an alarm signal. 

Owens 2002 teaches a system for processing video images to detect an event of 
interest (see title and figure 3), comprising: 

a generation device that receives the normal/abnormal signal and generates at 
least one active signal in response to the normal/abnormal signal indicating abnormal 
behavior of at least one track ("In abnormal sequences the novelty accumulator nodes 
are allowed to increase their activity, generating an alarm state" at abstract, last 
sentence; "alarm reporting as the event is developing" at page 1254, section 4, second 
paragraph, line 5). 

It would have obvious at the time the invention was made to one of ordinary skill 
in the art to utilize the alarm state signal of Owens 2002 to signify the unusual behavior 
of Owens 2000 such that crime prevention may occur by alerting a surveillance operator 
to the existence of possible suspicious behavior (see Owens 2002 at page 1254, 
second to last paragraph). 
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8. Claims 3, 7 and 8 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over the combination of Owens 2000 and Owens 2002 as applied to claim 1 above, and 
further in view of Schwerdt et al. ("Visual Recognition of Emotional States", ICMI). 

Regarding claim 3, the Owens 2000 and Owens 2002 combination discloses 
wherein the learning process accumulates data representing the behavior of the track(s) 
over a period of time using four dimensions representing x-position, y-position, x- 
velocity and y-velocity, of the track(s) within the video image ("flow vector, f = [x, y, dx. 
dy]" Owens 2000 at page 3, section 4, paragraph 3, line 1). 

The Owens 2000 and Owens 2002 combination does not disclose that the data is 
accumulated in a four-dimensional histogram. 

Schwerdt et al. discloses a method in the same field of endeavor of object 
tracking and recognition that accumulates data representing the behavior of the track(s) 
over a period of time in a four-dimensional histogram ("four dimensional histogram" at 
section 3.2, paragraph 4, line 1). 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to utilize the 4-D histogram of Schwerdt et al. to keep track of the mobile 
data of the Owens 2000 and Owens 2002 combination "because the memory space 
requirements are lower" (Schwerdt et al. at section 3.2, paragraph 6, line 8). 

Regarding claim 7, the Owens 2000, Owens 2002 and Schwerdt et al. 
combination discloses a method wherein the comparison process classifies a track 
according to a comparison of the frequency of occupation of a histogram cell 
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representing a corresponding position and velocity within the video images with an 
occupancy threshold (Schwerdt et al. at Table 2). 

Regarding claim 8, the Owens 2000, Owens 2002 and Schwerdt et al. 
combination discloses a method wherein the comparison process acts to classify as 
normal behavior a track adjacent or near a cell which is above the occupancy threshold, 
despite the track appearing in a cell below the occupancy threshold, where one cell is 
considered to be near another if the distance between them is below a predetermined 
distance threshold ("If this distance exceeds a threshold value then the feature vector is 
considered novel, and the trajectory is identified as suspicious" Owens 2000 at page 6, 
section 5, paragraph 7, line 5). 

9. Claim 1 3 is rejected under 35 U.S.C. 1 03(a) as being unpatentable over the 
combination of Owens 2000 and Owens 2002 as applied to claim 1 above, and further 
in view of Kamin (US 4,1 98,653). 

The Owens 2000 and Owens 2002 combination discloses the elements of claim 
1 as described in the 103 rejection above. 

The Owens 2000 and Owens 2002 combination does not disclose that 
subsequent active alarm signals are inhibited for a predetermined time interval after a 
first active alarm signal has been produced. 

Kamin discloses a method in the same field of endeavor of video tracking 
wherein subsequent active alarm signals are inhibited for a predetermined time interval 
after a first active alarm signal has been produced ("first alarm pulse A' (Fig. 3d), which 
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originates from field 2 and which would normally result in a spurious alarm, is 
suppressed at the right time" at col. 4, line 24). 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to utilize the delay of Kamin in the alarm generation device of the Owens 
2000 and Owens 2002 combination to provide "high sensitivity with respect to events 
which are relevant to a genuine alarm" (Kamin at col. 1 , line 59). 

10. Claim 4 is rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of Owens 2000, Owens 2002 and Schwerdt et al. as applied to claim 3 
above, and further in view of Franke et al. (US 6,41 1 ,328). 

The Owens 2000, Owens 2002 and Schwerdt et al. combination discloses a 
method wherein: 

the learn behavior stage segregates the tracks according to a velocity threshold 
("relatively low rates of velocity change are seen in normal pedestrian motion, while a 
pedestrian moving in between parked vehicles shows high rates of velocity change" 
Owens 2000 at page 3, section 4, paragraph 3, line 6); 

data concerning the mobile tracks is stored in said four-dimensional histogram 
(see claim 3 above). 

The Owens 2000, Owens 2002 and Schwerdt et al. combination does not 
disclose that tracks moving at a velocity below the velocity threshold are considered 
stationary while tracks moving at a velocity in excess of the velocity threshold are 
considered mobile. 
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Franke et al. teaches a method for a video motion anomaly detector for 
processing video images to detect an event of interest ("visible spectrum images may 
be used to detect traffic-related incidents" at col. 11, line 14), wherein: 

tracks moving at a velocity below the velocity threshold are considered stationary 
while tracks moving at a velocity in excess of the velocity threshold are considered 
mobile ("Slow vehicles can be identified in a time sequence of images by a method 
comprising estimating vehicle speed for each vehicle as above and identifying vehicles 
moving faster than a predetermined speed. Speeding vehicles can be identified by an 
analogous method in which vehicles moving faster than a predetermined speed are 
identified in the image sequence" at col. 8, line 59). 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to utilize the velocity distinction of Franke et al. to identify the tracks of the 
Owens 2000, Owens 2002 and Schwerdt et al. combination such that the operator may 
be warned of "the presence of substantially stationary objects in a relatively high-risk 
region of interest" (Franke et al. at col. 10, line 31) and the proper corrective action may 
occur thusly. 

The Owens 2000, Owens 2002, Schwerdt et al. and Franke et al. combination 
does not explicitly disclose that data concerning the stationary tracks is stored in a two- 
dimension histogram representing x-position and y-position within the video image. 

However, it is well-known in the art to utilize two-dimensional histograms to store 
position data and therefore would have been obvious at the time the invention was 
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made to store the stationary data of the Owens 2000, Owens 2002, Schwerdt et al. and 
Franke et al. combination in a two-dimensional histogram. 

1 1 . Claim 5 is rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of Owens 2000, Owens 2002 and Schwerdt et al. as applied to claim 3 
above, and further in view of Zuniga (US 5,546,474). 

The Owens 2000, Owens 2002 and Schwerdt et al. combination discloses the 
elements of claim 3 as described in the 103 rejection above. 

The Owens 2000, Owens 2002 and Schwerdt et al. combination does not 
disclose that a cell size of the histogram varies with speed. 

Zuniga discloses a method in the same field of endeavor of region detection 
wherein a cell size ("size of a cell can vary" at col. 10, line 14) of the histogram varies 
with speed ("moment of inertia" at col. 10, line 9). 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to utilize the cell varying of Zuniga in the histogram of the Owens 2000, 
Owens 2002 and Schwerdt et al. combination to allow the user to evaluate a 
"performance/quality tradeoff' (Zuniga at col. 10, line 15). 

12. Claim 6 is rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of Owens 2000, Owens 2002 and Schwerdt et al. as applied to claim 3 
above, and further in view of Jepson et al. (US 7,058,205). 
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The Owens 2000, Owens 2002 and Schwerdt et al. combination discloses the 
elements of claim 3 as described in the 103 rejection above. 

The Owens 2000, Owens 2002 and Schwerdt et al. combination does not 
disclose that the histogram is periodically de-weighted in order to bias the result of the 
learning process towards more recent events. 

Jepson et al. discloses a method in the same field of endeavor of motion tracking 
wherein the histogram is periodically de-weighted in order to bias the result of the 
learning process towards more recent events ("up-weight the more recent frames" at 
col. 3, line 19). 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to utilize the weight varying of Jepson et al. to the histogram of the Owens 
2000, Owens 2002 and Schwerdt et al. combination such that a "optimal motion 
estimation is achieved" (Jepson et al. at col. 3, line 12). 



Conclusion 



1 3. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to KATRINA FUJITA whose telephone number is 
(571)270-1574. The examiner can normally be reached on M-Th 8-5:30pm, F 8- 
4:30pm. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Brian Werner can be reached on (571) 272-7401 . The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



/Katrina Fujita/ 
Examiner, Art Unit 2624 



